Development and Validation of Novel Machine Learning-Based Risk Scores for Multiple Myeloma: Insights from the Harmony Alliance Big Data Platform

Introduction

Traditional risk stratification methods in Multiple Myeloma (MM) rely on clinical parameters and cytogenetic profiles, and often fall short in predictive accuracy. The advent of machine learning (ML) presents an opportunity to enhance risk prediction models by leveraging large datasets and complex variable interactions.

Objectives

The primary objective was to develop and validate three novel ML-driven prognostic models for newly diagnosed MM (NDMM) patients. These models aim to improve standard scores by incorporating a broader range of variables and a comprehensive statistical approach.

Methodology

We utilized data from the Harmony MM cohort, comprising 15,581 patients from various sources, including clinical trials within the European Myeloma Network. The core study was based on the same cohorts analyzed originally by the R2-ISS study, including 10,843 patients with NDMM enrolled in 16 clinical trials. An imputation technique was applied to enhance missing data annotation, including some completely missing in the R2-ISS cohort like hemoglobin, which was imputed from recently added trials and real-world registry data from both newly diagnosed and refractory patients (N=4,738). We implemented Random Forest (RF) survival models, focusing on maximizing the discriminative ability while minimizing the number of variables used. Variable importance guided dimensionality reduction was applied to eliminate redundant variables. Validation was conducted using out-of-bag cross-validation within the training set and external validation with Myeloma XI trial data, with accuracy metrics including c-indexes and time-dependent AUCs.

Results

A model based on the common 20 variables achieved a c-index of 0.666 in the training set and 0.667 in the test set for overall survival (OS) prediction, and 0.619 and 0.627 for progression-free survival (PFS) prediction. A simplified model using six variables (age, hemoglobin, B2-microglobulin, albumin, 1q gain, and 17p deletion) maintained high accuracy, with c-index scores of 0.655 in the training set and 0.664 in the test set for OS, and 0.609 and 0.620 respectively for PFS. Our model excluding cytogenetics achieved c-index scores of 0.637 in the training set and 0.648 in the test set for OS, and 0.596 & 0.614 respectively for PFS. Time-dependent AUCs confirmed the superiority and better reproducibility of the ML approaches compared with the ISS, R-ISS, and R2-ISS benchmarks. Furthermore, the ML models showed better generalization across different patient subsets, including both transplant-eligible and ineligible patients.

Then, we modeled OS and PFS outcomes including best treatment response, achieving out-of-bag c-index scores of 0.700 for OS and 0.701 for PFS in a subset of 6,518 patients with sufficient annotation. In order to extract treatment-related effects, we classified the diverse treatments into three main groups: IMID-based (3,078 patients), PI-based (1,139 patients), and PI-IMID-based (1,606 patients). This systematic grouping enabled us to discern treatment-related influences on OS and PFS outcomes, enhancing our understanding of their effectiveness. As a result, upfront treatment, 1q gain, and hemoglobin emerged as key predictors of PFS, underscoring their significance in patient prognosis.

Conclusion

Our study presents three novel ML prognostic models for NDMM that surpass traditional risk stratification methods. These models are accurate, generalizable, and functional with or without cytogenetic data, enhancing clinical utility. Effective in both transplant-eligible and ineligible patients, they underscore the role of AI in personalized medicine. Incorporating treatment response further enhances model performance. An online tool to calculate the score will be presented at ASH.

Mosquera Orgueira:Roche: Consultancy; Pfizer: Consultancy; Abbvie: Membership on an entity's Board of Directors or advisory committees, Speakers Bureau; AstraZeneca: Consultancy, Membership on an entity's Board of Directors or advisory committees, Speakers Bureau; Janssen: Consultancy, Membership on an entity's Board of Directors or advisory committees, Speakers Bureau; Takeda: Speakers Bureau; Biodigital THX: Current equity holder in private company; Novartis: Other; Incyte: Other; GSK: Consultancy. D'Agostino:GlaxoSmithKline: Honoraria, Membership on an entity's Board of Directors or advisory committees; Sanofi: Honoraria, Membership on an entity's Board of Directors or advisory committees; Bristol Myers Squibb: Membership on an entity's Board of Directors or advisory committees; Janssen: Honoraria, Research Funding; Adaptive Biotechnologies: Membership on an entity's Board of Directors or advisory committees. Larocca:Jansenn: Honoraria, Other: participation to advisory board; GSK: Honoraria, Other: participation to advisory board; Menarini: Honoraria, Other: participation to advisory board; Sanofi: Honoraria. Wester:Sanofi: Honoraria; Janssen: Honoraria. Salwender:Janssen: Honoraria, Other: Travel grant; Oncopeptides: Honoraria; Pfizer: Honoraria; Sanofi: Honoraria, Other: Travel grant; Stemline: Honoraria; Roche: Honoraria; Takeda: Honoraria; Chugai: Honoraria; GlaxoSmithKline: Honoraria; Bristol Myers Squibb/Celgene: Honoraria, Other: Travel grant; Amgen: Honoraria, Other: Travel grant; AbbVie: Honoraria; Sebia: Honoraria. Bringhen:AbbVie, Amgen, Bristol Myers Squibb, GlaxoSmithKline, Janssen, and Sanofi: Speakers Bureau; Sanofi: Consultancy, Honoraria; Bristol Myers Squibb, Janssen, Oncopeptides, Pfizer, Stemline Therapeutics, and Takeda: Other: Participation in advisory boards. Zweegman:Sanofi: Membership on an entity's Board of Directors or advisory committees; BMS: Membership on an entity's Board of Directors or advisory committees; GSK: Membership on an entity's Board of Directors or advisory committees; Janssen: Membership on an entity's Board of Directors or advisory committees, Research Funding; Amgen: Membership on an entity's Board of Directors or advisory committees; Takeda: Research Funding; Oncopeptides: Membership on an entity's Board of Directors or advisory committees. Hernandez Rivas:GlaxoSmithKline: Consultancy, Honoraria; Amgen: Honoraria, Membership on an entity's Board of Directors or advisory committees, Speakers Bureau; Pfizer: Honoraria, Membership on an entity's Board of Directors or advisory committees; Bristol Myers Squibb: Honoraria, Membership on an entity's Board of Directors or advisory committees, Research Funding, Speakers Bureau. Cook:Celgene: Research Funding; Amgen: Consultancy, Speakers Bureau; Janssen: Consultancy, Research Funding; Bristol Myers Squibb: Consultancy, Honoraria; Janssen-Cilag: Honoraria, Speakers Bureau; Takeda: Consultancy, Honoraria, Research Funding, Speakers Bureau. Kaiser:Pfizer: Consultancy, Honoraria; GSK: Consultancy; Sanofi: Consultancy; BMS/Celgene: Consultancy, Honoraria, Research Funding; Pfizer: Consultancy, Honoraria; J&J/Janssen: Consultancy, Honoraria, Research Funding; Roche: Consultancy; Poolbeg: Consultancy, Honoraria; Regeneron: Consultancy. Goldschmidt:Sanofi: Honoraria, Membership on an entity's Board of Directors or advisory committees, Other: Grants and/or provision of Investigational Medicinal Product; support for attending meetings and/or travel, Research Funding; Chugai: Honoraria, Other: Grants and/or provision of Investigational Medicinal Product; Adaptive Biotechnologies: Membership on an entity's Board of Directors or advisory committees; Millennium Pharmaceuticals Inc.: Research Funding; Takeda: Research Funding; Amgen: Honoraria, Membership on an entity's Board of Directors or advisory committees, Other: Support for attending meetings and/or travel; grants and/or provision of Investigational Medicinal Product, Research Funding; Celgene: Research Funding; Bristol Myers Squibb: Honoraria, Membership on an entity's Board of Directors or advisory committees, Other: Support for attending meetings and/or travel, Research Funding; Janssen: Honoraria, Membership on an entity's Board of Directors or advisory committees, Other: Grants and/or provision of Investigational Medicinal Product; support for attending meetings and/or travel, Research Funding; Karyopharm: Research Funding; Hoffmann-La Roche: Research Funding; Molecular Partners: Research Funding; Novartis: Honoraria, Other: Support for attending meetings and/or travel, Research Funding; MorphoSys AG: Research Funding; Bristol Myers Squibb/Celgene: Other: Grants and/or provision of Investigational Medicinal Product; Merck Sharp and Dohme (MSD): Research Funding; GlycoMimetics Inc.: Research Funding; Incyte Corporation: Research Funding; Dietmar Hopp Foundation: Other: Grants and/or provision of Investigational Medicinal Product; Array Biopharma/Pfizer: Other: Grants and/or provision of Investigational Medicinal Product; GlaxoSmithKline (GSK): Honoraria, Other: Support for attending meetings and/or travel, Research Funding; Heidelberg Pharma: Research Funding; Pfizer: Honoraria, Other: Support for attending meetings and/or travel, Research Funding; Johns Hopkins University: Other: Grants and/or provision of Investigational Medicinal Product; Mundipharma GmbH: Other: Grants and/or provision of Investigational Medicinal Product. Sonneveld:Oncopeptides: Patents & Royalties; Karyopharm: Membership on an entity's Board of Directors or advisory committees, Patents & Royalties, Research Funding; Pfizer: Membership on an entity's Board of Directors or advisory committees, Patents & Royalties; European Myeloma Network: Other: President; Celgene: Membership on an entity's Board of Directors or advisory committees, Research Funding; Janssen: Membership on an entity's Board of Directors or advisory committees, Patents & Royalties, Research Funding; Bristol Myers Squibb: Membership on an entity's Board of Directors or advisory committees, Research Funding; Amgen: Membership on an entity's Board of Directors or advisory committees, Research Funding. San-Miguel:Bristol Myers Squibb: Other: Advisory board; Celgene: Other: Advisory board; Roche: Other: Advisory board; Janssen-Cilag: Other: Advisory board; Novartis: Other; Karyopharm: Other: Advisory board; GlaxoSmithKline: Other: Advisory board; Haemalogix: Other: Advisory board; MSD: Other: Advisory board; Amgen: Consultancy, Other: Advisory Board ; Takeda: Other: Advisory board; Sanofi: Other: Advisory board; Abbvie: Consultancy, Other: Advisory Board; Regeneron: Other: Advisory board; SecuraBio: Other: Advisory board. Boccadoro:GlaxoSmithKline: Membership on an entity's Board of Directors or advisory committees; AbbVie: Honoraria; Bristol Myers Squibb: Honoraria, Research Funding; Janssen: Honoraria, Membership on an entity's Board of Directors or advisory committees, Research Funding; Novartis: Honoraria, Research Funding; Amgen: Honoraria, Research Funding; Celgene: Honoraria, Research Funding; Sanofi: Honoraria, Research Funding; Mundipharma: Research Funding. Mateos:BMS/Celgene, Janssen-Cilag, Sanofi, Abbvie, Stemline, Oncopeptides, GSK: Honoraria, Membership on an entity's Board of Directors or advisory committees; Amgen, Takeda, Regeneron: Honoraria.